Hierarchical Contrast for Unsupervised Skeleton-Based Action Representation Learning

This paper targets unsupervised skeleton-based action representation learning and proposes a new Hierarchical Contrast (HiCo) framework. Different from the existing contrastive-based solutions that typically represent an input skeleton sequence into instance-level features perform contrast holistically, our proposed HiCo represents multiple-level performs in hierarchical manner. Specifically, given human sequence, we it multiple feature vectors of different granularities both temporal spatial domains via sequence-to-sequence (S2S) encoders unified downsampling modules. Besides, is conducted terms four levels: instance level, domain clip part level. Moreover, orthogonal to S2S encoder, which allows us flexibly embrace state-of-the-art encoders. Extensive experiments on datasets, i.e., NTU-60, NTU-120, PKU-I PKU-II, show achieves for two downstream tasks including recognition retrieval, its learned good transferability. also framework effective semi-supervised recognition. Our code available at https://github.com/HuiGuanLab/HiCo.